
Human Research Program 

Space Human Factors & Habitability Element 

Space Human Factors Engineering Project (SHFE) 


Human-Robot Interaction 


Jennifer Rochlis Zumbado, Ph.D., NASA JSC, 

jennifer.l.rochlis@nasa.gov 

Aniko Sandor, Ph.D., Lockheed Martin 

aniko.sandor-1 @nasa.gov 

Neta Ezer, Ph.D., Futron Corporation 

neta.ezer@nasa.gov 


HRP Investigators Workshop 2012 

Houston, TX 



Background 


• Risk of Inadequate Design of Human and Automation/Robotic Integration 
(HARI) is a new Human Research Program (HRP) risk 

• HRI is a research area that seeks to understand the complex relationship 
among variables that affect the way humans and robots work together to 
accomplish goals 

• The DRP addresses three major HRI study areas that will provide 
appropriate information for navigation guidance to a teleoperator of a robot 
system, and contribute to the closure of currently identified HRP gaps 

- Overlays 

• Use of overlays for teleoperation to augment the information available on the video feed 

- Camera views 

• Type and arrangement of camera views for better task performance and awareness of 
surroundings 

- Command modalities 

• Development of gesture and voice command vocabularies 


Literature review of HRI methods and metrics 


• Completed literature review of methods and metrics in HRI and 
related fields (e.g., human-computer interaction, aviation, 
automation, and teamwork) 

• Goals were to identify comprehensive, valid, replicable, and practical 
research tools to study space HRI and identify research needs 

• Concentration on taxonomies of HRI, task and performance metrics, 
testbeds, measures of interaction, teamwork, communication, workload, 
situation awareness, and interface variables 

• Research areas applicable across space HRI 

• Compensations for poor or ambiguous video feedback 

• Match between command modality (e.g., gesture, voice, hand controller) 
and task 

• Communication issues (i.e., loss of signal, communication delay) and 
mitigations 



Literature review of overlays 


• Literature review on video overlays through augmented reality (AR) 

• Summarized research articles to understand the use of AR as guidance 
for operators during teleoperation and the effects of characteristics of 
AR on human task performance 

• Focused on navigation and robot arm alignment tasks 

• Type Of guidance (Foyle, Hooey, Wilson, & Johnson, 2002) 

• Command-guidance symbology gives operators direct information 
about the command inputs they need to make 

• Situation-guidance symbology gives operators cues to let them infer 
what command inputs are needed 

• Type of overlay 

• Superimposed symbology is fixed on the operators’ display and does 
not move with the video feed 

• Integrated symbology is linked to the scene in the video feed and 
moves with it 


Review of NASA robotic systems 
SPDM observations 


• Literature review 

• Current NASA robotic systems including the Shuttle and Station robotic arms, 
Mars rovers, the Special Purpose Dexterous Manipulator (SPDM) and Robonaut 
2 (R2) 

• SPDM procedures and operator handbooks 

• Observations and interviews 

• NASA robot system operations and interviews with NASA robotic operators and 
trainers 

• SPDM 

• Multi-mission Space Exploration Vehicle (MMSEV) 

• igoal laboratory gesture interface system 

• R2 


HRI workshop 


• Coordinated and held the Space HRI workshop November 14-17, 
2011 

• Nineteen subject matter experts from NASA centers, academia, and industry 
discussed and documented research areas needed to address the HARI risk 

• Discussions framed by classes of robotic systems and In relation to the HRP 
gaps 

• The research areas with the highest priority ratings: 

- Maintaining situation awareness 

- Applying research on human teams to human-robot teams 

- Knowledge of machine capabilities and states 

- Information sharing without overload 

- Variable time delay 

- Levels of automation 

- Changes in human performance over time 


Research Robot 



• CoroWare CoroBot for use in the FY12- 
FY14 studies 

• Programmable mobile robot, 12” x 13” x 16” 

• 4 degree-of-freedom robot arm 

• MS Kinect system for gesture interface 

• 4 cameras and RoboRealm computer vision software 
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